Endoscope apparatus and method of detecting edge

ABSTRACT

An endoscope apparatus includes a camera and circuitry. The camera is detachably connected to a base of an endoscope adapted to be inserted in a subject and includes a sensor configured to image a subject image captured by the endoscope. The circuitry is configured to process an image captured by the camera and generate a video signal for display, calculate a gain in white balance based on the captured image, and perform mask edge detecting processing for detecting a boundary point between the subject image and a mask area other than the subject image included in the captured image.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority to and incorporates by referencethe entire contents of Japanese Patent Application No. 2017-033937 filedin Japan on Feb. 24, 2017.

BACKGROUND

The present disclosure relates to an endoscope apparatus and a method ofdetecting an edge, the endoscope apparatus including an imaging deviceattachably and detachably connected to an eye contact portion of theendoscope inserted in a subject and having an imaging unit imaging thesubject image captured by the endoscope and a control device processingthe captured image obtained through imaging by the imaging unit andgenerating a video signal for display.

Known image processing apparatuses are known that process a capturedimage of the inside (the inside of a live body) of a subject such as ahuman body (for example, see Japanese Laid-open Patent Publication No.2015-134039). The image processing apparatus described in JapaneseLaid-open Patent Publication No. 2015-134039 obtains a captured imageincluding a subject image captured by an endoscope inserted into a livebody. A light (a subject image) captured by the endoscope has asubstantially circular sectional surface. The subject image in thecaptured image is therefore in a substantially circular shape. Thecaptured image includes a subject image and a mask area other than thesubject image. The image processing apparatus converts the capturedimage into a luminance image and detects a boundary point between thesubject image and the mask area using luminance distribution in theluminance image.

SUMMARY

In detecting a boundary point between a subject image and a mask area,if the subject is, for example, a white subject such as gauze, theluminance value of the subject image included in the captured image issufficiently high. This configuration allows more accurate detection ofthe boundary point between the subject image and the mask area using theluminance distribution on the captured image. On the other hand, indetecting a boundary point between a subject image and a mask area, useof another subject other than the above-described subject causes a greatdeal of variation among the luminance values on the subject imageincluded in the captured image. In this case, it is difficult toaccurately detect a boundary point between the subject image and themask area using the luminance distribution in the captured image. Thetechnique described in Japanese Laid-open Patent Publication No.2015-134039 specifies no subjects in detecting a boundary point betweena subject image and a mask area. This technique thus has difficulty inaccurately detecting a boundary point between a subject image and a maskarea.

An endoscope apparatus according to one aspect of the present disclosuremay include: an imaging device detachably connected to an eye contactportion of an endoscope adapted to be inserted in a subject, the imagingdevice including an imaging unit configured to image a subject imagecaptured by the endoscope; a control device configured to process animage captured by the imaging unit and generate a video signal fordisplay; an operation receiving unit configured to receive a useroperation for calculating a gain in white balance; a gain calculatingunit configured to calculate the gain in white balance based on thecaptured image in response to the user operation; and an edge detectingunit configured to perform mask edge detecting processing for detectinga boundary point between the subject image and a mask area other thanthe subject image included in the captured image in response to the useroperation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing that illustrates a schematic configuration of anendoscope apparatus according to an embodiment;

FIG. 2 is a block diagram that illustrates configurations of a camerahead and a control device;

FIG. 3 is a block diagram that illustrates a configuration of an edgeprocessing unit;

FIG. 4 is an illustrative drawing of mask edge detecting processing;

FIG. 5 is a flowchart that illustrates operation of the endoscopeapparatus; and

FIG. 6 is an illustrative drawing of operation of the endoscopeapparatus.

DETAILED DESCRIPTION

Embodiments (hereinafter, embodiments) will now be described withreference to the drawings. It should be noted that the followingembodiments are not intended to limit the scope of the presentdisclosure. In the drawings, like numerals indicate like components.

Schematic configuration of endoscope apparatus FIG. 1 is a drawing thatillustrates a schematic configuration of an endoscope apparatus 1according to an embodiment. The endoscope apparatus 1 is used in themedical field and observes the inside of a live body. As illustrated inFIG. 1, the endoscope apparatus 1 includes an insertion unit 2, a lightsupply device 3, a light guide 4, a camera head 5, a first transfercable 6, a display device 7, a second transfer cable 8, a control device9, and a third transfer cable 10.

The insertion unit 2 is constituted as a rigid endoscope. Morespecifically, the insertion unit 2 is rigid or at least partly flexiblein an elongated shape and is inserted in a live body. The inside of theinsertion unit 2 is provided with an optical system consisting of one ora plurality of lenses and gathering light of a subject image. The lightsupply device 3 has an end of the light guide 4 connected thereto andsupplies light for lighting the inside of the live body to the end ofthe light guide 4 under control of the control device 9. The light guide4 has an end attachably and detachably connected to the light supplydevice 3 and has the other end attachably and detachably connected tothe insertion unit 2. The light guide 4 transfers light supplied fromthe light supply device 3 from an end to the other end and supplies thelight to the insertion unit 2. The light supplied to the insertion unit2 is emitted from the front end of the insertion unit 2 and enters theinside of the live body. The light (the subject image) entering theinside of the live body and reflected in the live body is gathered bythe optical system in the insertion unit 2.

The camera head 5 functions as an imaging device according to thepresent disclosure. The camera head 5 is attachably and detachablyconnected to the base end (an eye contact portion 21 in FIG. 1) of theinsertion unit 2. The camera head 5 captures the subject image gatheredby the insertion unit 2 in the form of a light and outputs an imagesignal (RAW signal) obtained through the imaging under control of thecontrol device 9. Examples of the image signal include a 4K or higherimage signal. The configuration of the camera head 5 will be describedlater in detail.

The first transfer cable 6 has an end attachably and detachablyconnected to the control device 9 through a connector CN1 (FIG. 1) andhas the other end attachably and detachably connected to the camera head5 through a connector CN2 (FIG. 1). The first transfer cable 6 transfersan image signal and others output from the camera head 5 to the controldevice 9 and transfers a control signal, a synchronization signal, aclock, electric power, and others output from the control device 9 tothe camera head 5. The camera head 5 may transfer an image signal andothers to the control device 9 through the first transfer cable 6 usingan optical signal or an electric signal. The control device 9 maytransfer a control signal, a synchronization signal, and a clock to thecamera head 5 through the first transfer cable 6 in the same manner.

The display device 7 is constituted as a liquid crystal display, anorganic electro luminescence (EL) display, or the like and displays animage based on a video signal transmitted from the control device 9under control of the control device 9. The second transfer cable 8 hasan end attachably and detachably connected to the display device 7 andhas the other end attachably and detachably connected to the controldevice 9. The second transfer cable 8 transfers a video signal processedby the control device 9 to the display device 7.

The control device 9 includes a central processing unit (CPU) and othersand integrally controls operation of the light supply device 3, thecamera head 5, and the display device 7. The configuration of thecontrol device 9 will be described later in detail. The third transfercable 10 has an end attachably and detachably connected to the lightsupply device 3 and has the other end attachably and detachablyconnected to the control device 9. The third transfer cable 10 transfersa control signal received from the control device 9 to the light supplydevice 3.

Configuration of Camera Head

The configuration of the camera head 5 will now be described. FIG. 2 isa block diagram that illustrates configurations of the camera head 5 andthe control device 9. In FIG. 2, the connectors CN1 and CN2 respectivelyconnecting between the control device 9 and the first transfer cable 6and connecting between the camera head 5 and the first transfer cable 6,connectors respectively connecting between the control device 9 and thesecond transfer cable 8 and connecting between the display device 7 andthe second transfer cable 8, and connectors respectively connectingbetween the control device 9 and the third transfer cable 10 andconnecting between the light supply device 3 and the third transfercable 10 are not illustrated for convenience. As illustrated in FIG. 2,the camera head 5 includes a lens unit 51, a lens driving unit 52, alens position detecting unit 53, an imaging unit 54, and a communicationunit 55.

The lens unit 51 is constituted of a plurality of lenses movable alongan optical axis and forms a subject image, the light of which isgathered by the insertion unit 2, on an imaging surface of the imagingunit 54 (an imaging element 541 (FIG. 2)). As illustrated in FIG. 2, thelens unit 51 includes a focus lens 511 and a zoom lens 512. The focuslens 511 is constituted of one of or a plurality of lenses and adjuststhe focal point by moving along the optical axis. The zoom lens 512 isconstituted of one of or a plurality of lenses and adjusts the angle ofview by moving along the optical axis. The lens unit 51 is furtherprovided with a focus mechanism (not illustrated) moving the focus lens511 along the optical axis and an optical zoom mechanism (notillustrated) moving the zoom lens 512 along the optical axis. Asillustrated in FIG. 2, the lens driving unit 52 includes a motor 521operating the above-described focus mechanism and optical zoom mechanismand a driver 522 driving the motor 521. The lens driving unit 52 adjuststhe focal point and the angle of view of the lens unit 51 under controlof the control device 9.

The lens position detecting unit 53 is constituted as a position sensorsuch as a photointerrupter and detects the lens position (hereinafter,the focus position) of the focus lens 511 and the lens position(hereinafter, the zoom position) of the zoom lens 512. The lens positiondetecting unit 53 outputs a detection signal based on the focus positionand the zoom position to the control device 9 through the first transfercable 6.

The imaging unit 54 images the inside of a live body under control ofthe control device 9. As illustrated in FIG. 2, the imaging unit 54includes the imaging element 541 and a signal processing unit 542. Theimaging element 541 is constituted as a charge coupled device (CCD), acomplementary metal oxide semiconductor (CMOS), or the like thatreceives a subject image gathered by the insertion unit 2 in the form ofa light and formed by the lens unit 51 and converts the light to anelectric signal (analog signal). The signal processing unit 542 performssignal processing on the electric signal (analog signal) received fromthe imaging element 541 and outputs an image signal (RAW signal (digitalsignal)). For example, the signal processing unit 542 performs, on theelectric signal (analog signal) from the imaging element 541, signalprocessing such as processing of removing reset noise, processing ofmultiplying the analog signal by an analog gain for amplification of theanalog signal, and analog to digital conversion.

The communication unit 55 functions as a transmitter transmitting animage signal (RAW signal (digital signal)) output from the imaging unit54 to the control device 9 through the first transfer cable 6. Thecommunication unit 55 is constituted as a high-speed serial interfacethat transmits an image signal to the control device 9 at a transferrate of one Gbps or greater through the first transfer cable 6.

Configuration of Control Device

The configuration of the control device 9 will now be described withreference to FIG. 2. As illustrated in FIG. 2, the control device 9includes a communication unit 91, an image processing unit 92, awave-detection processing unit 93, an edge processing unit 94, a displaycontrol unit 95, a control unit 96, an input unit 97, an output unit 98,and a storage unit 99. The communication unit 91 functions as a receiverthat receives an image signal (RAW signal (digital signal)) output fromthe camera head 5 (the communication unit 55) through the first transfercable 6. The communication unit 91 is constituted as a high-speed serialinterface that receives an image signal from the communication unit 55at a transfer rate of one Gbps or greater.

The image processing unit 92 processes the image signal (RAW signal(digital signal)) output from the camera head 5 (the communication unit55) and received by the communication unit 91 under control of thecontrol unit 96. For example, the image processing unit 92 multipliesthe image signal (RAW signal (digital signal)) by a digital gain foramplification of the digital signal. The image processing unit 92performs RAW processing such as optical black subtraction processing anddemosaic processing on the image signal (RAW signal (digital signal))multiplied by the digital gain and converts the RAW signal (imagesignal) to an RGB signal (image signal). The image processing unit 92further performs RGB processing such as white balance adjustmentprocessing for multiplying RGB values by respective gains, RGB gammacorrection, and YC conversion (converting an RGB signal (image signal)to a luminance signal and a color difference signal (Y, C_(B)/C_(R)signal)) on the RGB signal. The image processing unit 92 furtherperforms YC processing such as color difference correction and noisereduction on the Y, C_(B)/C_(R) signal (image signal).

The wave-detection processing unit 93 inputs the image signal (Y,C_(B)/C_(R) signal) processed by the image processing unit 92 andperforms wave-detection processing based on the image signal (Y,C_(B)/C_(R) signal). For example, based on pixel information (aluminance signal (Y signal)) of each pixel in a certain area(hereinafter, a wave-detection area) in the whole captured image in oneframe captured by the imaging element 541, the wave-detection processingunit 93 carries out detection of contrast and frequency components ofthe image in the wave-detection area, detection of the average luminancevalue and the maximum and minimum pixels in the wave-detection areausing a filter and the like, comparison with a threshold, and detectionof a histogram and others. The wave-detection processing unit 93 outputswave-detection information (such as contrast, a frequency component, theaverage luminance value, the maximum and minimum pixels, and ahistogram) obtained through the detection to the control unit 96.

FIG. 3 is a block diagram that illustrates a configuration of the edgeprocessing unit 94. The edge processing unit 94 performs mask edgedetecting processing and first and second determination processing basedon a luminance signal (Y signal) constituting the image signal (Y,C_(B)/C_(R) signal) processed by the image processing unit 92. Asillustrated in FIG. 3, the edge processing unit 94 includes an edgedetecting unit 941, an extracting unit 942, a processing possibilitydetermining unit 943, a pixel position recognizing unit 944, and avariation determining unit 945.

The edge detecting unit 941 performs the following mask edge detectingprocessing in response to a user operation (hereinafter, white balancesetting operation) to the input unit 97 for having a gain (a gain usedin the white balance adjustment processing performed by the imageprocessing unit 92) in white balance calculated according to a result ofdetermination of the first determination processing performed by theprocessing possibility determining unit 943. FIG. 4 is an illustrativedrawing of mask edge detecting processing. More specifically, (a) ofFIG. 4 is a drawing that illustrates an exemplary captured image CIcaptured by the imaging element 541. (b) of FIG. 4 is a drawing thatillustrates distribution of luminance values on a horizontal line L5 inthe captured image CI of (a) of FIG. 4. A light (a subject image)reflected in a live body and gathered into the insertion unit 2 has asubstantially circular sectional surface. A subject image SI in thecaptured image CI is thus substantially circular as illustrated in (a)of FIG. 4. The captured image CI includes the subject image SI and amask area (the black area in (a) of FIG. 4) MA other than the subjectimage SI. The edge detecting unit 941 performs mask edge detectingprocessing and detects a boundary point BP ((a) of FIG. 4) between thesubject image SI and the mask area MA.

More specifically, as illustrated in (a) of FIG. 4, the edge detectingunit 941 acquires a luminance signal (Y signal) of image signals (Y,C_(B)/C_(R) signals) processed by the image processing unit 92. The edgedetecting unit 941 detects distribution of luminance values on each of aplurality of (14 lines in this embodiment) horizontal lines L1 to L14 inthe captured image CI based on the luminance signal (Y signal). On thecaptured image CI, the area of the subject image SI has higher luminancevalues than those in the mask area MA. As illustrated in (b) of FIG. 4,in regard to the luminance distribution on the horizontal line L5, theluminance values are high in the part between two boundary points BP,each of which is set between the subject image SI and the mask area MA,and low in other parts. The edge detecting unit 941 compares theluminance values with a first luminance threshold SB1 ((b) of FIG. 4)and recognizes an area having contiguous pixels with higher luminancevalues than the first luminance threshold SB1 as the subject image SI.The edge detecting unit 941 further compares the luminance values with asecond threshold SB2 ((b) of FIG. 4) lower than the first luminancethreshold SB1 and recognizes an area having contiguous pixels with lowerluminance values than the second luminance threshold SB2 as the maskarea MA. The edge detecting unit 941 recognizes the boundary point BP((a) of FIG. 4) between the subject image SI and the mask area MA. Theedge detecting unit 941 performs this processing on each of thehorizontal lines L1 to L14 and recognizes a plurality of boundary pointsBP between the subject image SI and the mask area MA.

The extracting unit 942 acquires a luminance signal (Y signal) of imagesignals (Y, C_(B)/C_(R) signals) processed by the image processing unit92. The extracting unit 942 compares the luminance value of each pixelon the horizontal line L7 ((a) of FIG. 4) positioned in the center ofthe captured image CI with the first luminance threshold SB1 based onthe luminance signal (Y signal) and extracts first pixels with higherluminance values than the first luminance threshold SB1. The extractingunit 942 further compares each luminance value on the horizontal line L7with the second luminance threshold SB2 and extracts second pixels withlower luminance values than the second luminance threshold SB2.

The processing possibility determining unit 943 performs the followingfirst determination processing. The processing possibility determiningunit 943 compares the number of first pixels (hereinafter, a firstcontiguous pixel number N1 (see FIG. 6) extracted by the extracting unit942 and contiguously aligned on the horizontal line L7 with a firstpixel number threshold SN1 (see (a) of FIG. 6) and determines whetherthe image is in a processing possible state that the first contiguouspixel number N1 is equal to or greater than the first pixel numberthreshold SN1 or in a processing impossible state that the firstcontiguous pixel number N1 is less than the first pixel number thresholdSN1. The processing possibility determining unit 943 further comparesthe number of second pixels (hereinafter, a second contiguous pixelnumber N2 (see FIG. 6) extracted by the extracting unit 942 andcontiguously aligned on the horizontal line L7 with a second pixelnumber threshold SN2 (see (a) of FIG. 6) and determines whether theimage is in a processing possible state that the second contiguous pixelnumber N2 is equal to or greater than the second pixel number thresholdSN2 or in a processing impossible state that the second contiguous pixelnumber N2 is less than the second pixel number threshold SN2. Theprocessing possibility determining unit 943 outputs a detection signalaccording to the result of determination in the first determinationprocessing to the control unit 96.

When the processing possibility determining unit 943 determines theimage to be in a processing possible state, the pixel positionrecognizing unit 944 recognizes the pixel positions of second pixels inwhich the second contiguous pixel number N2 is equal to or greater thanthe second pixel number threshold SN2. The variation determining unit945 performs the following second determination processing. Thevariation determining unit 945 determines whether all the pixels in thepixel positions recognized by the pixel position recognizing unit 944are contiguously extracted as the second pixels by the extracting unit942 after the mask edge detecting processing by the edge detecting unit941.

The display control unit 95 generates a video signal for display wherean image (which corresponds to warning information according to thepresent disclosure) including a message such as “retry white balancesetting operation” is overlapped on the captured image CI based on theimage signal (Y, C_(B)/C_(R) signal) processed by the image processingunit 92 by using, for example, on-screen display (OSD) processing undercontrol of the control unit 96. The display control unit 95 outputs thevideo signal to the display device 7 through the second transfer cable8. In other words, the display control unit 95 and the display device 7function as warning information notifying units 100 (FIG. 2) accordingto the present disclosure.

The control unit 96 is constituted of a CPU and others. The control unit96 controls operation of the light supply device 3, the camera head 5,and the display device 7 by outputting control signals through the firstto the third transfer cables 6, 8, and 10 and integrally controlsoperation of the control device 9. As illustrated in FIG. 2, the controlunit 96 includes a lens control unit 961, a parameter calculating unit962, and brightness control unit 963. The lens control unit 961 operatesthe lens driving unit 52 and adjusts (changes the focus position and thezoom position) the focal point and the angle of view of the lens unit51. For example, the lens control unit 961 calculates a focusingevaluation value for evaluating a focusing status of the subject imageSI included in the captured image CI based on wave-detection information(such as contrast and frequency components) output from thewave-detection processing unit 93. The lens control unit 961 determinesthe contrast detected by the wave-detection processing unit 93 or thesum of high frequency components of the frequency components detected bythe wave-detection processing unit 93 to be a focusing evaluation value.A greater focusing evaluation value indicates more accurate focusing.The lens control unit 961 thereafter performs AF processing forpositioning the focus lens 511 in a focus position where the subjectimage SI comes to be in a focusing status using, for example, the hillclimbing based on the focus position detected by the lens positiondetecting unit 53 and the focusing evaluation value. The AF processingmay be continuous AF in which the processing is continuously performedor one-touch AF in which the processing is performed in response tooperation of an operation button (not illustrated) provided to thecamera head 5 or the like.

The parameter calculating unit 962 calculates a brightness parameter forchanging brightness of the captured image CI obtained through imaging bythe imaging unit 54 to reference brightness (changing the averageluminance value obtained in the wave-detection processing to a referenceaverage luminance value) based on the wave-detection information (theaverage luminance value) output from the wave-detection processing unit93. In this embodiment, the parameter calculating unit 962 calculatesfour brightness parameters including an exposure time of each pixel ofthe imaging element 541, an analog gain multiplied by the signalprocessing unit 542, a digital gain multiplied by the image processingunit 92, the amount of light supplied from the light supply device 3 tothe insertion unit 2 based on the wave-detection information (theaverage luminance value) output from the wave-detection processing unit93. The parameter calculating unit 962 further calculates respectivegains by which the RGB values are multiplied in the white balanceadjustment processing performed by the image processing unit 92 based onthe wave-detection information output from the wave-detection processingunit 93. The parameter calculating unit 962 outputs a control signal tothe image processing unit 92 and determines the calculated gains to begains by which the RGB values are multiplied in the white balanceadjustment processing performed by the image processing unit 92. Theparameter calculating unit 962 functions as a gain calculating unitaccording to the present disclosure.

The brightness control unit 963 controls operation of the imagingelement 541, the signal processing unit 542, the image processing unit92, and the light supply device 3 based on the brightness parameterscalculated by the parameter calculating unit 962. More specifically, thebrightness control unit 963 outputs a control signal to the imaging unit54 through the first transfer cable 6 and determines the exposure time(a brightness parameter) calculated by the parameter calculating unit962 to be the exposure time of each pixel of the imaging element 541.The brightness control unit 963 outputs a control signal to the imagingunit 54 through the first transfer cable 6 and determines the analoggain (a brightness parameter) calculated by the parameter calculatingunit 962 to be the analogue gain multiplied by the signal processingunit 542. The brightness control unit 963 outputs a control signal tothe image processing unit 92 and determines the digital gain (abrightness parameter) calculated by the parameter calculating unit 962to be the digital gain multiplied by the image processing unit 92.Furthermore, the brightness control unit 963 outputs a control signal tothe light supply device 3 through the third transfer cable 10 anddetermines the amount of light (a brightness parameter) calculated bythe parameter calculating unit 962 to be the amount of light suppliedfrom the light supply device 3 to the insertion unit 2. As describedabove, the brightness control unit 963 controls operation of the imagingelement 541, the signal processing unit 542, the image processing unit92, and the light supply device 3, whereby brightness of the capturedimage CI is changed to the reference brightness.

The input unit 97 is constituted as an operation device such as a mouse,a keyboard, and a touch panel and receives a user operation (forexample, white balance setting operation) of a user, for example, adoctor. The input unit 97 outputs an operation signal according to theuser operation to the control unit 96. The input unit 97 functions as anoperation receiving unit according to the present disclosure. The outputunit 98 is constituted as a speaker, a printer, and others and outputsvarious kinds of information. The storage unit 99 stores a programexecuted by the control unit 96, information necessary for processingperformed by the control unit 96, and others.

Operation of Endoscope Apparatus

Operation (a method of detecting an edge) of the above-describedendoscope apparatus 1 will now be described. FIG. 5 is a flowchart thatillustrates operation of the endoscope apparatus 1. FIG. 6 is anillustrative drawing of operation of the endoscope apparatus 1. Morespecifically, FIG. 6 is a drawing that illustrates distribution ofluminance values on the horizontal line L7 positioned in the center ofthe captured image CI illustrated in (a) of FIG. 4. Operation of theedge processing unit 94, the wave-detection processing unit 93, thedisplay control unit 95, and the display device 7 will be mainlydescribed. A user, for example, a doctor puts a white subject such asgauze on the front end of the insertion unit 2 and arranges the subjectin the field of view of the insertion unit 2 (Step S1: subject arrangingstep). The endoscope apparatus 1 starts imaging the subject (Step S2:imaging step).

After Step S2, the control unit 96 continuously monitors whether theinput unit 97 receives the white balance setting operation (Step S3:operation receiving step). If the control unit 96 determines that nowhite balance setting operation is received (No at Step S3), the controlunit 96 continues monitoring at Step S3. If the control unit 96determines that the white balance setting operation has been received(Yes at Step S3), the extracting unit 942 acquires a luminance signal (Ysignal) of image signals (Y, C_(B)/C_(R) signals) processed by the imageprocessing unit 92. The extracting unit 942 compares the luminance valueof each pixel on the horizontal line L7 positioned in the center of thecaptured image CI with the first luminance threshold SB1 based on theluminance signal (Y signal) and extracts first pixels having a higherluminance value than the first luminance threshold SB1 (Step S4).

After Step S4, the processing possibility determining unit 943 comparesthe first contiguous pixel number N1 in which the first pixels extractedat Step S4 are contiguously aligned on the horizontal line L7 with thefirst pixel number threshold SN1 and determines whether the firstcontiguous pixel number N1 is equal to or greater than the first pixelnumber threshold SN1 (whether the image is in a processing possiblestate or in a processing impossible state) (Step S5). The processingpossibility determining unit 943 thereafter outputs a signal accordingto the result of determination to the control unit 96.

(a) of FIG. 6 illustrates distribution of luminance values on thehorizontal line L7 in the captured image CI with the insertion unit 2connected to the camera head 5 and with a white subject arranged in thefield of view of the insertion unit 2 at Step S1 (hereinafter referredto as a first state). In the first state, the subject image SI has asufficiently high luminance value with the first contiguous pixel numberN1 equal to or greater than the first pixel number threshold SN1. Theimage is therefore determined to be in a processing possible state atStep S5. (b) of FIG. 6 illustrates distribution of luminance values onthe horizontal line L7 in the captured image CI with the insertion unit2 connected to the camera head 5 and with no white subjects arranged inthe field of view of the insertion unit 2 at Step S1 (hereinafterreferred to as a second state). In the second state, the subject imageSI has a great deal of variation among the luminance values with thefirst contiguous pixel number N1 less than the first pixel numberthreshold SN1. The image is therefore determined to be in a processingimpossible state at Step S5. Furthermore, (c) of FIG. 6 illustratesdistribution of luminance values on the horizontal line L7 in thecaptured image CI with no insertion units 2 connected to the camera head5 (hereinafter referred to as a third state). In the third state, withthe insertion unit 2 detached from the camera head 5, the entirecaptured image CI has a great deal of variation among the luminancevalues. As is the case with the second state, the image is determined tobe in a processing impossible state at Step S5 with the first contiguouspixel number N1 less than the first pixel number threshold SN1.

If the processing possibility determining unit 943 determines that thefirst contiguous pixel number N1 is less than the first pixel numberthreshold SN1 (the image is in a processing impossible state) (No atStep S5), the apparatus may possibly be in the second or the thirdstate. The boundary point BP thus may not be accurately detected in themask edge detecting processing. The display control unit 95 has thedisplay device 7 display a display image where an image (warninginformation) including a message such as “retry white balance settingoperation” is overlapped on the captured image CI under control of thecontrol unit 96 (Step S6). The endoscope apparatus 1 returns to Step S3.

If the processing possibility determining unit 943 determines that thefirst contiguous pixel number N1 is equal to or greater than the firstpixel number threshold SN1 (the image is in a processing possible state)(Yes at Step S5), the extracting unit 942 acquires a luminance signal (Ysignal) of image signals (Y, C_(B)/C_(R) signals) processed by the imageprocessing unit 92. The extracting unit 942 compares the luminance valueof each pixel on the horizontal line L7 positioned in the center of thecaptured image CI with the second luminance threshold SB2 based on theluminance signal (Y signal) and extracts second pixels with a lowerluminance value than the second luminance threshold SB2 (Step S7). Inthe subsequent steps, Step S7 is continuously performed in parallel withother processing.

After Step S7, the processing possibility determining unit 943 comparesthe second contiguous pixel number N2 in which the second pixelsextracted at Step S7 are contiguously aligned on the horizontal line L7with the second pixel number threshold SN2 and determines whether thesecond contiguous pixel number N2 is equal to or greater than the secondpixel number threshold SN2 (whether the image is in a processingpossible state or in a processing impossible state) (Step S8). Theprocessing possibility determining unit 943 thereafter outputs a signalaccording to the result of determination to the control unit 96.

In the first and the second states, the insertion unit 2 is connected tothe camera head 5. As illustrated in (a) and (b) of FIG. 6, the maskarea MA has a sufficiently low luminance value with the secondcontiguous pixel number N2 equal to or greater than the second pixelnumber threshold SN2. The image is therefore determined to be in aprocessing possible state at Step S8. In the third state, the insertionunit 2 is not connected to the camera head 5. As illustrated in (c) ofFIG. 6, the captured image CI has a great deal of variation among theluminance values with the second contiguous pixel number N2 less thanthe second pixel number threshold SN2. The image is therefore determinedto be in a processing impossible state at Step S8.

If the processing possibility determining unit 943 determines that thesecond contiguous pixel number N2 is less than the second pixel numberthreshold SN2 (the image is in a processing impossible state) (No atStep S8), the apparatus may possibly be in the third state. The boundarypoint BP thus may not be accurately detected in the mask edge detectingprocessing. The endoscope apparatus 1 therefore moves back to Step S6.If the processing possibility determining unit 943 determines that thesecond contiguous pixel number N2 is equal to or greater than the secondpixel number threshold SN2 (the image is in a processing possible state)(Yes at Step S8), the pixel position recognizing unit 944 recognizes thepixel position (each pixel position in the mask area MA) of each secondpixel in which the second contiguous pixel number N2 used fordetermination in Step S8 is equal to or greater than the second pixelnumber threshold SN2 (Step S9).

After Step S9, the edge detecting unit 941 performs the mask edgedetecting processing (Step S10: mask edge detecting step). After StepS10, the wave-detection processing unit 93 acquires luminance signals (Ysignals) of image signals (Y, C_(B)/C_(R) signals) processed by theimage processing unit 92. The wave-detection processing unit 93determines the area of the subject image SI surrounded by a plurality ofboundary points BP detected at Step S9 to be a wave-detection area. Thewave-detection processing unit 93 performs wave-detection processingbased on a luminance signal (Y signal) of each pixel in thewave-detection area of the acquired luminance signals (Y signals) (StepS11) and outputs wave-detection information obtained by thewave-detection processing to the control unit 96. In the subsequentsteps, Step S11 is continuously performed in parallel with otherprocessing.

After Step S11, the parameter calculating unit 962 calculates respectivegains by which the RGB values are multiplied in the white balanceadjustment processing performed by the image processing unit 92 based onthe wave-detection information output from the wave-detection processingunit 93 (Step S12: gain calculating step). The parameter calculatingunit 962 outputs a control signal to the image processing unit 92 anddetermines the calculated gains to be gains by which the RGB values aremultiplied in the white balance adjustment processing performed by theimage processing unit 92. The parameter calculating unit 962 furthercalculates brightness parameters based on the wave-detectioninformation.

After Step S12, the variation determining unit 945 determines whetherall the pixels in the pixel positions recognized at Step S9 arecontiguously extracted as the second pixels by the extracting unit 942after Step S10 (Step S13). The variation determining unit 945 outputs asignal according to the result of determination to the control unit 96.If the apparatus transitions to the second state after undergoing themask edge detecting processing in the first state, which means that thewhite subject has been merely removed from the front end of theinsertion unit 2, there is no changes in the luminance value of eachpixel in the mask area MA as seen from comparison between (a) of FIG. 6and (b) of FIG. 6. Step S13 is therefore determined to be “Yes”. If theapparatus transitions to the third state after undergoing the mask edgedetecting processing in the first state, which means that the insertionunit 2 has been detached from the camera head 5, the entire capturedimage CI comes to have a great deal of variation among the luminancevalues. As seen from comparison between (a) of FIG. 6 and (c) of FIG. 6,some pixels in the pixel positions in the mask area MA recognized in thefirst state are not extracted as the second pixels (their luminancevalues are equal to or greater than the second luminance threshold SB2).Step S13 is therefore determined to be “No”.

If the pixels are determined not to be contiguously extracted as thesecond pixels (No at Step S13), the apparatus has transitioned from thefirst state to the third state, and the insertion unit 2 may possibly bereplaced by a different insertion unit 2 (for example, such an insertionunit 2 that has a different diameter and accordingly provides a subjectimage SI in the captured image CI having a different diameter). Theendoscope apparatus 1 therefore moves back to Step S6. On the otherhand, if the pixels are determined to be contiguously extracted as thesecond pixels (Yes at Step S13), the endoscope apparatus 1 continuesStep S13.

The above-described embodiment exerts the following advantageouseffects. The present disclosure is based on an action of a user, forexample, a doctor, who uses a white subject such as gauze when having anendoscope apparatus 1 calculate a gain in white balance. The endoscopeapparatus 1 according to the embodiment performs the mask edge detectingprocessing that detects a boundary point BP between a subject image SIand a mask area MA based on a luminance signal of each pixel in thecaptured image CI in response to the white balance setting operation.The endoscope apparatus 1 is able to perform the mask edge detectingprocessing in the first state in which a white subject such as gauze isused. With this process, the subject image SI included in the capturedimage CI has a sufficiently high luminance value, which allows moreaccurate detection of the boundary point BP between the subject image SIand the mask area MA using the luminance distribution in the capturedimage CI.

The endoscope apparatus 1 according to the embodiment determines whetherthe image is in a processing possible state or in a processingimpossible state based on comparison between the luminance value andeach of the first and the second luminance thresholds SB1 and SB2,comparison between the first contiguous pixel number N1 and the firstpixel number threshold SN1, and comparison between the second contiguouspixel number N2 and the second pixel number threshold SN2. The endoscopeapparatus 1 performs the mask edge detecting processing only when theimage is determined to be in a processing possible state. With thisconfiguration, the endoscope apparatus 1 may avoid performing the maskedge detecting processing in the second state, in which a subject otherthan a white subject such as gauze is used, and in the third state, inwhich the insertion unit 2 is detached from the camera head 5. In otherwords, the endoscope apparatus 1 may achieve effects of performing themask edge detecting processing only in the first state and accuratelydetecting the boundary point BP between the above-described subjectimage SI and mask area MA in a preferred manner.

Furthermore, if the image is in a processing impossible state, theendoscope apparatus 1 according to the embodiment displays a displayimage where an image including a message such as “retry white balancesetting operation” is overlapped on the captured image CI. This processenables the user such as a doctor to recognize that the apparatus is inthe second or the third state. In other words, this process allows theapparatus to transition from the second or the third state to the firststate and is able to have the user such as a doctor retry the whitebalance setting operation. The mask edge detecting processing may beperformed in the first state, which may achieve effects of accuratelydetecting the boundary point BP between the above-described subjectimage SI and mask area MA in a preferred manner.

If the insertion unit 2 is replaced by another insertion unit 2 afterthe mask edge detecting processing and the calculation processing forcalculating a gain in white balance, the calculation processing forcalculating a gain in white balance needs to be performed againconsidering the individual variability between the insertion units 2.Furthermore, the insertion units 2 may have respective differentdiameters, and in this case, the mask edge detecting processing alsoneeds to be performed again. The endoscope apparatus 1 according to theembodiment recognizes the pixel position of second pixels in which thesecond contiguous pixel number N2 is equal to or greater than the secondpixel number threshold SN2 after determination of the image to be in aprocessing possible state and determines whether all the pixels in therecognized pixel positions are contiguously extracted as the secondpixels (the second determination processing). Upon determination of“No”, the endoscope apparatus 1 displays a display image where an imageincluding a message such as “retry white balance setting operation” isoverlapped on the captured image CI. The second determination processingallows determination of whether the apparatus has transitioned from thefirst state to the third state and whether the insertion unit 2 is in astate to be replaced by another insertion unit 2. The endoscopeapparatus 1 is further able to have a user such as a doctor retry thewhite balance setting operation by displaying a display image where animage including a message such as “retry white balance settingoperation” is overlapped on the captured image CI. The endoscopeapparatus 1 is able to perform the calculation processing forcalculating a gain in white balance and the mask edge detectingprocessing again on the replaced insertion unit 2.

The endoscope apparatus 1 according to a first embodiment performswave-detection processing on a wave-detection area, which is the area ofthe subject image SI surrounded by a plurality of boundary points BPbetween the subject image SI detected by the mask edge detectingprocessing and the mask area MA. This configuration allows thewave-detection processing to be performed on the largest possiblewave-detection area (substantially the entire area of the subject imageSI) excluding the mask area MA. Processing (for example, AF processingand calculating processing for calculating a brightness parameter) basedon wave-detection information obtained by the wave-detection processingmay be therefore accurately performed.

Other Embodiments

The embodiment has been described; however, it should be noted that thepresent disclosure is not limited to the above-described embodiment. Inthe above-described embodiment, at least a part of the configuration(the lens unit 51, the lens driving unit 52, the lens position detectingunit 53, and the imaging unit 54) provided to the camera head 5 may beprovided to the front end in the insertion unit 2. The insertion unit 2is not limited to a rigid endoscope and may be a flexible endoscope. Inthe above-described embodiment, at least a part of the functions of thecontrol unit 96 may be provided outside (to the camera head 5, theconnectors CN1 and CN2) the control device 9. In the above-describedembodiment, the operation receiving unit receiving the white balancesetting operation according to the present disclosure is not necessarilyprovided to the control device 9 and may be provided to the camera head5. In the above-described first embodiment, the warning informationnotifying unit 100 displays an image including a message such as “retrywhite balance setting operation”; however, the method is not limitedthereto. The warning information notifying unit 100 may notify the userof the information by another method (for example, output the message invoice). In the above-described embodiment, the endoscope apparatus 1 maybe used in the industrial field and observe the inside of a subject suchas a mechanical structure.

The present disclosure is based on an action of a user, for example, adoctor, who uses a white subject such as gauze when having the endoscopecalculate a gain in white balance. An endoscope apparatus according tothe present disclosure performs mask edge detecting processing thatdetects a boundary point between a subject image and a mask areaincluded in a captured image based on a luminance signal of each pixelin the captured image in response to a user operation for having a gainin white balance calculated. The endoscope apparatus is able to performthe mask edge detecting processing when the subject is a white subjectsuch as gauze. This structure significantly increases the luminancevalue of the subject image included in the captured image, which allowsmore accurate detection of a boundary point between the subject imageand the mask area using the luminance distribution on the capturedimage. Moreover, as is the case of the above-described endoscopeapparatus, a method of detecting an edge according to the presentdisclosure is based on the above-described action of a user such as adoctor and exerts the same advantageous effects as those of theabove-described endoscope apparatus.

Although the disclosure has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. An endoscope apparatus comprising: a cameradetachably connected to a base of an endoscope adapted to be inserted ina subject, the camera including a sensor configured to image a subjectimage captured by the endoscope; and circuitry configured to: process animage captured by the camera and generate a video signal for display;receive a user operation for calculating a gain in white balance;calculate the gain in white balance based on the captured image inresponse to the user operation; compare a luminance value based on aluminance signal of each pixel on a line in the captured image with afirst luminance threshold and extract first pixels having a hitherluminance value than the first luminance threshold; compare a firstcontiguous pixel number in which the first pixels are contiguouslyaliened on the line with a first pixel number threshold and determinewhether the captured image is in a processing possible state that thefirst contiguous pixel number is equal to or greater than the firstpixel number threshold or in a processing impossible state that thefirst contiguous pixel number is less than the first pixel numberthreshold; and on condition that the image is in the processing possiblestate, perform mask edge detecting processing for detecting a boundarypoint between the subject image and a mask area other than the subjectimage included in the captured image in response to the user operation.2. The endoscope apparatus according to claim 1, wherein the circuitryis further configured to: compare a luminance value based on a luminancesignal of each pixel on a line in the captured image with a secondluminance threshold and extracts second pixels having a lower luminancevalue than the second luminance threshold; compare a second contiguouspixel number in which the second pixels are contiguously aligned on theline with a second pixel number threshold and determines whether thecaptured image is in a processing possible state that the secondcontiguous pixel number is equal to or greater than the second pixelnumber threshold or in a processing impossible state that the secondcontiguous pixel number is less than the second pixel number threshold;and perform the mask edge detecting processing on condition that theimage is in the processing possible state.
 3. The endoscope apparatusaccording to claim 1, wherein the circuitry is further configured to:issue warning information when the captured image is determined to be inthe processing impossible state.
 4. The endoscope apparatus according toclaim 1, wherein the circuitry is further configured to: compare aluminance value based on a luminance signal of each pixel on a line inthe captured image with a second luminance threshold and extracts secondpixels having a lower luminance value than the second luminancethreshold; recognize a pixel position of the second pixel in which asecond contiguous pixel number having the second pixels contiguouslyaligned on the line is equal to or greater than a second pixel numberthreshold; determine whether all pixels in the pixel positions arecontiguously extracted as the second pixels; and issue warninginformation when the pixels are not contiguously extracted as the secondpixels.
 5. The endoscope apparatus according to claim 1, wherein thecircuitry is further configured to: perform wave-detection processingfor calculating a brightness parameter used for changing brightness onthe captured image based on a luminance signal of each pixel in an areaon the captured image surrounded by the boundary point.
 6. The endoscopeapparatus according to claim 1, wherein the circuitry is furtherconfigured to: perform wave-detection processing for controlling thecamera based on a luminance signal of each pixel in an area on thecaptured image surrounded by the boundary point.
 7. A method ofdetecting an edge executed by the endoscope apparatus according to claim1, the method comprising: arranging a white subject in a field of viewof the endoscope; receiving a user operation for calculating a gain inwhite balance; imaging a subject image captured by the endoscope;calculating the gain in white balance based on the image captured duringthe imaging in response to the user operation; and detecting a boundarypoint between the subject image and a mask area other than the subjectimage included in the captured image based on a luminance signal of eachpixel in the captured image in response to the user operation.
 8. Anendoscope apparatus comprising: a camera detachably connected to a baseof an endoscope adapted to be inserted in a subject, the cameraincluding a sensor configured to image a subject image captured by theendoscope; and circuitry configured to: process an image captured by thecamera and generate a video signal for display; compare a luminancevalue based on a luminance signal of each pixel on a line in thecaptured image with a first luminance threshold and extract first pixelshaving a higher luminance value than the first luminance threshold;compare a first contiguous pixel number in which the first pixels arecontiguously aligned on the line with a first pixel number threshold anddetermine whether the captured image is in a processing possible statethat the first contiguous pixel number is equal to or greater than thefirst pixel number threshold or in a processing impossible state thatthe first contiguous pixel number is less than the first pixel numberthreshold; and on condition that the image is in the processing possiblestate, perform mask edge detecting processing for detecting a boundarypoint between the subject image and a mask area other than the subjectimage included in the captured image.
 9. An endoscope apparatuscomprising: a camera detachably connected to a base of an endoscopeadapted to be inserted in a subject, the camera including a sensorconfigured to image a subject image captured by the endoscope; andcircuitry configured to: process an image captured by the camera andgenerate a video signal for display; receive a user operation forcalculating a gain in white balance; calculate the gain in white balancebased on the captured image in response to the user operation; performmask edge detecting processing for detecting a boundary point betweenthe subject image and a mask area other than the subject image includedin the captured image; and perform wave-detection processing forcalculating a brightness parameter used for changing brightness on thecaptured image based on a luminance signal of each pixel in an area onthe captured image surrounded by the boundary point in response to theuser operation.
 10. A method of detecting an edge executed by theendoscope apparatus according to claim 9, the method comprising:arranging a white subject in a field of view of the endoscope; receivinga user operation for calculating a gain in white balance; imaging asubject image captured by the endoscope; calculating the gain in whitebalance based on the image captured during the imaging in response tothe user operation; and detecting a boundary point between the subjectimage and a mask area other than the subject image included in thecaptured image based on a luminance signal of each pixel in the capturedimage in response to the user operation.
 11. An endoscope apparatuscomprising: a camera detachably connected to a base of an endoscopeadapted to be inserted in a subject, the camera including a sensorconfigured to image a subject image captured by the endoscope; andcircuitry configured to: process an image captured by the camera andgenerate a video signal for display; perform mask edge detectingprocessing for detecting a boundary point between the subject image anda mask area other than the subject image included in the captured image;and perform wave-detection processing for calculating a brightnessparameter used for changing brightness on the captured image based on aluminance signal of each pixel in an area on the captured imagesurrounded by the boundary point.